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FOREWORD 


This  advanced  development  effort  was  conducted  under  program  element  0603720N 
(Education  and  Training),  project  work  unit  R1772,  task  ET003  (Skills  Enhancement  Program).  It 
was  sponsored  by  the  Chief  of  Naval  Operations.  The  objectives  of  the  work  unit  were  to  develop 
guidelines  for  identifying  the  Navy  school  requirements  for  enhanced  student  preparation  and 
training  support,  and  methods  for  addressing  these  requirements. 

The  objective  of  the  present  effort  was  to  establish  an  index  of  training  productivity  for  use  in 
evaluating  Navy  training  programs  based  on  currently  available  criteria. 


THOMAS  F.  FINLEY 
Captain,  U.S.  Navy 
Commanding  Officer 


RICHARD  C.  SORENSON 
Technical  Director  (Acting) 
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SUMMARY 


Background 

Large  training  organizations  such  as  the  U.S.  Navy  need  to  maintain  a  constant  watch  over  the 
productivity  of  their  training  programs.  Training  productivity  reflects  a  school’s  training  outputs  in 
terms  of  the  quantity  of  graduates  and  the  level  to  which  they  are  trained,  as  well  as  the  training 
inputs  needed  to  maintain  training  effectiveness.  While  a  variety  of  criteria  contribute  to  training 
productivity,  no  method  is  presently  available  for  combining  these  criteria  into  a  single  overall 
productivity  index. 

Objective 

The  objective  of  this  effort  was  to  develop  an  index  of  productivity  that  combines  major 
productivity  criteria,  now  available  from  Navy  data  bases.  These  criteria  include  graduation/ 
attrition  rates,  setback  rates,  and  course  test  scores. 

Approach 

Formulas  for  combining  graduation  rates,  setback  rates,  and  test  scores  into  a  single  index  of 
productivity  were  developed  and  applied  to  recent  training  data  from  four  Navy  ratings.  Using 
these  formulas,  yearly  values  of  achieved  productivity  were  computed  for  each  rating  for  each  of 
several  years  of  instruction. 

Results 

Resulting  values  of  achieved  productivity  were  found  to  reflect  combined  changes  in  the  values 
of  individual  criteria  in  a  reasonable  and  logical  manner. 

Conclusion 

The  index  of  achieved  productivity  is  an  effective  device  for  monitoring  the  training 
productivity  of  Navy  technical  schools. 

Recommendation 

An  index  of  achieved  productivity  should  be  maintained  by  an  appropriate  Navy  training 
activity  and  reported  to  training  administrators  to  assist  them  in  monitoring  the  productivity  of 
technical  schools. 
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INTRODUCTION 


Background 

Large  training  organizations  such  as  the  U.S.  Navy  need  to  maintain  a  constant  watch  over  the 
productivity  of  their  training  programs.  Training  productivity  reflects  the  quantity  and  quality  of  a 
school’s  training  output  in  terms  of  the  number  of  graduates  arid  the  level  to  which  they  are  trained. 
Training  productivity  also  reflects  the  resources  required  to  achieve  ouqiuts  such  as  time  and  costs. 
Training  productivity  can  be  increased  by  raising  outputs  or  decreasing  inputs. 

Different  indices  of  training  productivity  are  monitored  by  individual  school  commands  and 
reviewed  by  higher  level  Navy  training  organizadons  responsible  for  the  allocation  of  training 
resources.  Training  producdvity  is  a  major  concern  to  such  organizadons  since  decreases  in 
training  productivity  can  inflate  training  costs  and  fail  to  meet  fleet  manpower  needs  for  qualified 
school  graduates  who  can  perform  effectively  on  the  job.  Schools  with  lew  productivity  may 
require  more  resources  or  longer  course  lengths  in  order  to  meet  their  training  requirements. 

Maintaining  training  productivity  is  not  the  whole  story.  The  advent  of  the  Total  Quality 
Leadership  program  into  the  Navy  has  emphasized  the  need  for  measures  of  productivity  that  can 
be  used  to  improve  the  quality  of  products.  In  Navy  training,  the  product  consists  of  school 
graduates  and  the  quality  of  that  product  depends  on  the  quality  of  the  training  provided.  The  Navy 
is  constandy  revising  tests  and  course  work,  implementing  motivational  techniques,  and 
introducing  new  training  methods  and  technologies  to  improve  training. 

While  the  need  to  monitor  Navy  training  productivity  may  be  readily  recognized,  the  criteria 
that  should  be  used  in  determining  productivity  are  less  obvious.  Historically,  the  criteria  most 
frequently  used  by  Navy  administrators  in  determining  or  comparing  the  training  productivity  of 
Navy  schools  have  been  student  attrition  or  graduation  rates  and  setback  rates.  One  reason  for  this 
is  the  ready  availability  of  such  data.  Numbers  of  students,  attrites,  graduates,  and  setbacks  in  Navy 
schools  have  been  recorded  and  maintained  in  Navy  data  bases  for  over  a  decade. 

However,  attrition/graduation  rates  and  setback  rates  only  provide  part  of  the  productivity 
picture.  Attrition/graduation  rates  are  indications  of  training  quantity.  Setback  rates  are  indications 
of  training  resource  requirements  (in  terms  of  training  time).  Neither  of  these  criteria  provide  an 
indication  of  the  quality  of  training  outputs.  An  indication  of  training  quality  is  readily  available, 
however,  in  the  form  of  course  test  scores.  Course  test  scores  may  be  considered  as  indexes  of 
training  quality  in  that  they  indicate  the  proportion  of  the  course  work  that  the  students  have 
mastered.  Presumably,  students  who  have  mastered  course  objectives  will  perform  better  on  the 
job.  If  this  is  not  the  case,  the  existing  training  objectives  are  inappropriate  and  should  be  revised. 

Until  recent  times,  records  of  course  test  scores  were  only  available  to  the  schools  that  recorded 
them  since  they  were  not  maintained  in  Navy  data  bases.  However,  such  records  can  now  be 
obtained  for  any  Navy  school  that  is  supported  by  the  Instructional  Support  System  (ISS). 
Established  in  1984,  ISS  is  maintained  by  the  Navy  Education  and  Training  Program  Management 
Support  Activity  (NETPMSA). 
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As  an  index  of  productivity,  course  test  scores  are  useful  in  their  own  right.  Unlike  attrition  and 
setback  rates,  which  may  be  increased  or  decreased  by  school  administrators  without  regard  for 
how  well  students  are  performing,  changes  in  course  test  scores  directly  reflect  student 
achievement. 

Because  training  productivity  depends  on  multiple  criteria,  it  is  important  to  consider  all 
relevant  criteria  in  conducting  course  evaluations.  Otherwise,  one  criterion  may  be  enhanced  at  the 
exjrense  of  the  others.  In  such  a  case,  training  effectiveness  may  appear  to  be  improved  when  in 
reality  one  training  problem  has  merely  been  substituted  for  another.  An  excellent  example  of  such 
a  situation  is  depicted  in  Table  1 . 


Table  1 

Changes  in  Radioman  (RM)  Productivity  Data 


Year 

Setback 

Rate 

Graduation 

Rate 

Average  Test 
Score 

1986 

25% 

84% 

90% 

1987 

8% 

76% 

90% 

1988 

0% 

77% 

92% 

1989 

24% 

85% 

93% 

Data  in  Table  1  indicate  that,  from  1986  to  1987,  setbacks  were  reduced  from  2.5  to  8  percent, 
a  major  improvement.  During  the  same  period,  however,  the  graduation  rate  decreased  from  84  to 
76  percent.  Later,  the  low  number  of  graduates  was,  apparently,  a  cause  for  concern  and,  from  1988 
to  1989,  the  graduation  rate  was  raised  from  77  to  85  percent,  another  major  improvement.  At  the 
same  time,  however,  setbacks  increased  from  0  to  24  percent. 

These  data  indicate  that  neither  lowering  the  setback  rates  in  1987  nor  raising  the  graduation 
rates  in  1989  produced  real  gains  in  training  productivity'.  In  fact,  during  both  of  these  periods,  test 
scores  remained  fairly  stable.  What  actually  occurred  was  that  one  index  was  improved  at  the 
expense  of  another. 

In  this  example,  course  test  scores  did  not  vary  as  a  function  of  graduation  or  setback  rates. 
Under  other  conditions,  however,  we  might  expect  test  scores  to  interact  with  other  productivity 
criteria.  For  example,  we  would  expect  that  raising  student  achievement  on  tests  would  lead  to 
higher  graduation  rates  since,  with  higher  scores,  fewer  students  would  fail  course  sections. 
Conversely,  increases  in  academic  attrition  might  actually  raise  test  score  averages.  This  could 
occur  if  the  lower  scoring  students  were  eliminated  early  in  the  course  and  did  not  participate  in 
later  course  secti'^ns. 

Since  graduation  rates,  test  scores,  and  training  time  can  interact  and  vary  at  the  same  time,  it 
is  often  difficult  to  determine  what  such  changes  mean.  Are  the  training  outcomes  positive. 
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negative,  or  neutral?  Have  we  gained  or  lost  productivity?  It  would  be  useful  in  such  circumstances 
to  be  able  to  combine  these  criteria  into  a  single  overall  productivity  index. 

Objective 

The  objective  of  this  effort  was  to  develop  an  index  of  productivity  that  combines  major 
productivity  criteria,  now  available  from  Navy  data  bases.  These  criteria  include  graduation/ 
attrition  rates,  setback  rates,  and  course  test  scores. 

APPROACH 

Establishing  Productivity  Fo.-mulas 

Overall  Productivity 

The  first  consideration  in  the  development  of  productivity  formulas  was  how  to  combine 
graduation  rates,  setback  rates,  and  test  scores  into  a  single  index  of  overall  productivity.  This  goal 
was  achieved  through  the  use  of  the  formula; 

Po  =  (G  X  S)/(Tc  +  T,), 

where  is  overall  productivity,  G  is  the  graduation  rate  (the  percentage  of  students  who 
graduate),  S  is  the  average  test  score  for  the  course  (the  percentage  of  correct  answers),  T^.  is  the 
course  length  (in  days  of  instruction),  and  Tj  is  the  number  of  additional  days  of  instruction  due 
to  setbacks. 

This  combined  index  Pq  has  several  advantages.  First  of  all,  an  increase  in  one  factor  at  the 
expense  of  another  will  limit  overall  gains.  For  example,  if  more  students  are  graduated  without 
increasing  student  test  scores  or  providing  more  training  time,  average  test  scores  should  fall  since 
the  lower  scoring  students  are  being  retained.  In  such  a  case,  the  increase  in  Pq  due  to  the  higher 
percentage  of  graduates  will  be  compensated  for  by  a  decrease  in  Pq  caused  by  lower  test  scores. 
In  a  similar  manner,  attempting  to  increase  the  number  of  graduates  by  allowing  more  setbacks  to 
occur  would  have  opposing  effects  on  Pq.  The  increase  in  the  number  c/‘  graduates  would  raise  Pq 
but  the  increase  in  total  training  time  from  the  increased  setbacks  would  lower  it.  A  second 
advantage  of  the  index  Pq  is  that  its  extreme  values  are  appropriate  in  a  mathematical  sense.  For 
example,  as  the  graduation  rate  (G)  and  the  average  course  test  score  (S)  each  approach  KX) 
percent,  the  value  G  x  S  approaches  1(X)  percent.  If  either  G  or  S  approach  0  percent,  then  G  x  S 
approaches  0  percent.  These  are  reasonable  outcomes  since  the  value  of  u^ining  is  only  maximized 
if  both  training  quantity  and  quality  are  maximized,  but  training  is  worthless,  either  if  no  one 
graduates  or  nothing  is  learned. 

Still  another  advantage  of  Pq  is  that  it  translates  setbacks  into  training  time  (Tg).  Tj  added  to  T^. 
(course  length)  represents  total  training  time  for  the  course.  This,  in  turn,  allow'S  graduation  rates 
and  test  scores  to  be  examined  in  relation  to  the  total  amount  of  training  time  provided  to  students. 


To  determine  Tj  for  a  given  course,  three  values  are  needed:  the  number  cf  students,  the  number 
of  setbacks,  and  the  average  duration  in  days  of  setback  periods.  Values  for  all  three  can  be  readily 
obtained  from  ISS.  ISS  data  include  setback  periods  for  each  student.  These  can  be  averaged  to 
determine  the  average  length  of  setback  per  student  (in  days).  T^  can  then  be  calculated  by 
multiplying  the  average  setback  length  by  the  number  of  setbacks  and  div  iding  by  the  total  number 
of  students. 

T-  -  average  setback  length  in  davs  x  No.  of  setbacks 
total  No.  of  students 

Table  2  presents  examples  of  calculated  values  of  Tj  and  the  data  on  which  these  calculations 
were  based. 

Table  2 

Setback  Data  for  the  Radioman  (RM)  “A”  School 


Year 

No.  of 
Students 

No.  of 
Setbacks 

Setback 

Length 

Setback  Time 

Per  Student  (T^) 

1986 

2,956 

700 

8  days 

1.9  days 

1987 

2,695 

223 

9  days 

0.7  days 

1988 

2,567 

0 

Odays 

0.0  days 

1989 

2,430 

574 

15  days 

3.5  days 

Tj  is  a  useful  index,  in  its  own  right,  for  comparing  trade-offs  between  setback  time  and  course 
length.  Generally  speaking,  if  the  length  of  a  course  is  increased  without  changing  the  curriculum, 
we  would  expect  setback  time  to  decrease  since,  with  more  time  to  study,  students  should  achieve 
higher  test  scores  and  require  less  setback  time.  This,  in  turn,  raises  the  possibility  that  increases  in 
course  length  could  actually  decrease  overall  training  time  by  reducing  the  number  of  setbacks.  As 
a  hypothetical  example,  suppose  that,  at  a  school,  setbacks  are  found  to  add  an  average  of  10  days 
of  ttu  ning  time  per  student.  Further,  suppose  thet  setbacks  could  be  eliminated  by  adding  7  days 
rc  tbe  'ength  of  the  course.  In  such  a  case,  the  addition  of  7  days  in  course  length  would  eliminate 
10  lays  of  setback  time  and  actually  reduce  total  training  time  by  3  days. 

1  laving  calculated  amounts  of  setback  time  (T^)  (as  displayed  in  Table  2),  we  can  use  our 
formula  to  obtain  values  of  Pq  for  the  RM  “A”  school.  Graduation  rates  (G)  and  average  test  scores 
(S)  can  be  obtained  from  Table  1.  T^.  (the  normal  length  of  the  RM  course)  is  65  days.  Inserting 
these  values  into  our  productivity  formula  produces  the  following  values  of  Pq  for  the  period  from 
1986  through  1989; 

1986:  Pq  =  (84%  x  907c)/(65  +  1.9)  days  =  1.13%  per  day 

1987:  Pq  =  (76%  x  907o)/(65  +  ,7)  days  =  1.04%  per  day 

1988:  Pq  =  (77%  x  92%)/(65  +  .0)  days  =  1.08%  per  day 

1989:  Pq  =  (85%  x  93%)/(65  +  3.5)  days  =1.15%  per  day. 


These  values  of  Pq  combine  the  effects  of  graduation  rates,  test  scores,  and  setback  times.  In 
their  present  form,  however,  they  are  of  limited  use.  They  can  be  used  to  compare  productivity 
achieved  during  different  time  periods,  but  they  provide  little  insight  in  terras  of  the  significance 
of  the  levels  achieved.  Is  the  1989  RM  “A”  school  productivity  level  of  1.15  percent  high?  Is  the 
1987  level  of  1.04  percent  low?  It’s  difficult  to  say.  A  more  useful  index  of  productivity  can  be 
obtained  by  expressing  overall  productivity  levels  as  percentages  of  potential  overall  productivity. 

Potential  Overall  Productivity 

Potential  overall  productivity  (Pp)  is  defined  here  as  the  highest  level  of  productivity  that  can 
be  obtained  with  existing  course  lengths.  The  highest  possible  level  of  productivity  occurs  when 
the  graduation  rate  (G)  is  100  percent,  the  average  course  test  score  (S)  is  100  percent,  and  the  rate 
of  setbacks  (Tj)  equals  zero  so  that  total  training  time  equals  T^,  the  length  of  the  course. 

Pp  can  be  calculated  with  the  same  formula  used  to  calculate  overall  productivity  (Pq),  by 
substituting  100  percent  for  G  and  S,  and  0  days  for  T^: 

Pp  =  (GxS)/(Tc  +  Ts) 

=  (100%  X  100%)/(Tc  +  0)  =  100%/Tc. 

Since  Tc  (normal  course  length)  for  the  RM  “A”  school  is  65  days,  the  potential  overall 
productivity  of  the  RM  “A"  school  is: 

Pp  =  l(X)%/65  days  =  1 .54%  per  day. 


Achieved  Productivity 

Having  developed  an  index  of  potential  productivity  (Pp),  it  now  becomes  possible  to  consider 
overall  productivity  (Pq)  (the  productivity  achieved)  as  a  percentage  of  potential  productivity.  The 
achieved  percentage  of  potential  productivity  is  designated  here  as  and  is  expressed  as; 

Pa  ' 

That  is.  achieved  productivity  equals  overall  productivity  divided  by  potential  overall 
productivity. 

Applying  Productivity  Formulas 

To  demonstrate  the  utility  of  an  index  of  achieved  productivity,  these  productivity  formulas 
were  applied  to  obtain  values  of  achieved  productivity  based  on  recent  student  performance  in 
several  Navy  schools. 


RESULTS 

To  determine  whether  tlie  proposed  index  of  achieved  productivity  provides  a  meaningful 
summary  of  available  productivity  data,  we  obtained  values  of  achieved  productivity  based  on 
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recent  student  performance  in  several  Navy  technical  schools.  Analyses  were  performed  for  the 
RM  “A”  school  students  and  for  “A”  school  students  from  several  other  Navy  ratings  that  are 
monitored  by  ISS  and  have  experienced  major  changes  in  graduation  rates,  test  scores,  or  setback 
rates  in  recent  years.  These  include  Quartermaster  (QM),  Aviation  Control  Technician  (AQ),  and 
Avionics  Technician  (AT)  students.  Values  of  achieved  productivity  were  obtained  only  for  the 
years  in  which  major  changes  occurred  in  total  training  times  (Tj.  +  T^),  graduation  rates  (G),  or 
average  test  scores  (S). 

Obtained  values  of  achieved  productivity  are  presented  in  Thble  3  with  the  data  on  which  they 
arc  based.  Data  in  Table  3  for  the  RM  students  indicate  moderately  high  levels  of  productivity 
achievement  with  the  highest  level  in  1989.  These  levels  appear  reasonable  considering  that  test 
scores  and  graduation  rates  were  also  moderately  high  during  these  periods.  Both  test  scores  and 
graduation  rates  peaked  in  1989,  resulting  in  the  highest  productivity  level  for  the  four-year  period.  • 

However,  productivity  for  1989  was  less  than  it  might  have  been  due  to  the  increase  in  the  number 
of  setback  days  that  occurred  between  1988  and  1989  (see  Table  2). 

Data  in  Table  3  indicate  a  sizeable  increase  in  achieved  productivity  for  QM  students  between 
1986  and  1987.  The  gain  in  productivity  appears  reasonable  since  graduation  rates  and  course  test 
scores  both  increased  while  training  time  decreased.  Some  of  these  gains  in  productivity  were  lost 
in  1989,  despite  a  slight  increase  in  test  scores.  These  test  score  increases  were  more  than  offset  by 
a  decreased  graduation  rate  and  longer  training  time. 

Data  iti  Table  3  indicate  large  drops  in  achieved  prtKluciivity  for  both  AQ  and  AT  students 
between  1988  and  1989.  The  magnitude  of  the  losses  appears  reasonable  in  that  all  three 
productivity  criteria  combined  to  lower  productivity.  Graduation  rates  and  test  scores  decreased 
while  total  training  lime  increased  during  tlicse  periods.  AQ  and  AT  students  show  similar 
productivity  patterns  since  students  in  both  ratings  train  together  at  the  same  “A”  school. 

DISCUSSION 

We  have  demonstrated  how  a  single  index  can  provide  a  meaningful  assessment  of  achieved 
productivity  using  available  productivity  data.  Consideration  will  now  be  given  to  the  use  of  this 
achieved  productivity  index,  its  applications,  and  its  limitations. 

The  primary  advantage  of  this  index  is  that  it  accounts  for  interactions  among  productivity 
criteria.  When  simultaneous  changes  occur  in  different  productivity  criteria,  this  index  indicates  if 
the  combined  effect  of  the  changes  represents  an  actual  improvement  in  training  effectiveness  or 
if  we  have  merely  sacrificed  one  productivity  criterion  for  another. 

This  index  could  be  particularly  useful  to  tho.se  responsible  for  monitoring  the  overall 
effectiveness  of  Navy  technical  training.  The  index  can  be  used  to  compare  the  productivity  of 
differep'  Navy  schools.  This  is  possible  despite  differences  in  the  difficulty  of  their  training  content 
or  course  length,  because  all  schools  are  compared  in  terms  of  their  potential  achievement.  Such 
comparisons  could  be  used  as  a  basis  for  assigning  schools  priorities  for  training  assistance. 


6 


Table  3 


“A**  School  Productivity  Data  for  Selected  Ratings 


Year 

Total  Training 

Time  (T^.  +  Tj)® 

Grad. 

Rate 

Average 

Score 

Achieved 

Productivity 

Radioman  Students 

1986 

66.9  days 

84% 

90% 

73% 

1987 

65.7  days 

76% 

90% 

68% 

1988 

65.0  days 

77% 

92% 

71% 

1989 

68.5  days 

85% 

93% 

75% 

Quartermaster  Students 

1986 

21.5  days 

89% 

87% 

76% 

1987 

21.1  days 

94% 

89% 

83% 

1988 

21.0  days 

93% 

90% 

84% 

1989 

21.2  days 

90% 

91% 

81% 

Aviation  Control  Technician  Students 

1988 

1 10.0  days 

86% 

89% 

77% 

1989 

1 12.0  days 

76% 

84% 

63% 

Avionics  Technician  Students 

1988 

1 10.1  days 

90% 

90% 

81% 

1989 

112.5  days 

82% 

86% 

69% 

*Tc  =  Courv:  length. 

Tg  >  Setback  time. 

Yearly  summaries  of  achieved  productivity  could  be  used  to  identify  training  problems  that 
need  to  be  adiiressed.  When  the  achieved  productivity  of  any  Navy  school  slips  below  prescribed 
levels,  conectivc  measures  can  be  taken.  When  a  school's  achieved  productivity  approaches  the 
higher  levels  of  potential  achievement,  excessive  training  time  or  resources  may  be  expended. 
Under  such  conditions,  it  may  be  desirable  to  shorten  course  lengths  or  reduce  training  resources. 
When  efforts  are  made  to  improve  training,  the  same  index  can  provide  a  basis  for  assessing  the 
effectiveness  of  the  improvements.  Because  of  the  comprehensive  orientation  of  the  index,  it  will 
be  clear  whether  training  effectiveness  has  actually  improved  or  whether  one  productivity  criterion 
has  been  sacrificed  for  another. 

Several  considerations  should  be  kept  in  mind  in  applying  the  index.  It  is  probably  only 
meaningful  to  mal;e  productivity  co.niparisons  over  relatively  long  time  periods.  Factors  that  affect 
training  effectiveness,  such  as  student  ability,  instructor  experience,  student/instructor  ratios,  and 
classroom  habitability,  can  vary-  considerably  from  month  to  month.  Over  longer  time  periods,  such 
variations  tend  to  average  out.  It  is  important,  then,  to  maintain  productivity  records  for  Navy 
schools  over  time  so  that  an  established  baseline  of  data  is  available  before  schools  are  compared 
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or  new  training  methods  are  evaluated.  By  the  same  token,  sufficient  time  should  be  taken  in 
monitoring  productivity  under  a  revised  method  of  instruction  so  that  temporary  effects  can  be 
ameliorated  and  training  can  occur  under  a  range  of  conditions  large  enough  to  provide  stable  data. 

It  may  be  desirable  to  assign  weights  to  component  criteria.  While  no  weights  were  assigned 
to  criteria  in  the  present  study.  Navy  administrators  may  wish  to  emphasize  some  criteria  over 
others.  They  might,  for  example,  consider  graduation  rates  more  important  than  test  scores  or 
training  dme.  More  or  less  emphasis  can  easily  be  placed  on  any  component  criteria  by  assigning 
the  variable  a  weight;  that  is,  a  multiplier  to  increase  or  decrease  the  contribution  of  the  variable  to 
the  overall  productivity  ratio. 

Another  consideration  in  determining  achieved  productivity  (Pg)is  the  accuracy  of  the  recorded 
data.  Accuracy  of  data  is  a  concern  regardless  of  the  productivity  criteria  that  are  used.  However, 
data  accuracy  is  a  more  important  problem  when  several  different  types  of  data  are  combined.  For 
example,  if  5  percent  of  the  data  entries  are  missing  for  each  of  three  criteria,  as  many  as  15  percent 
of  the  entries  could  be  missing  at  least  one  of  the  three  types  of  data.  Such  considerations 
emphasize  the  need  to  ensure  that  complete  and  accurate  records  are  maintained. 

Still  another  consideration  relates  to  ensuring  the  equivalency  of  test  scores  used  in  obtaining 
values  of  achieved  productivity.  While  test  scores  are  independent  of  administrative  decisions  to 
raise  or  lower  passing  criteria,  they  are  not  immune  to  changes  or  modifications  to  course  tests. 
Changes  to  course  tests  should  ht  recorded  and  records  of  the  changes  maintained  so  that 
investigators  will  know  if  test  data  from  different  lime  periods  can  be  compared.  Also,  different 
versions  of  tests  are  sometimes  given  to  students.  Therefore,  for  comparisons  of  productivity  to  be 
valid,  the  different  test  versions  used  in  the  comparisons  must  be  equated  in  terms  of  content  and 
level  of  difficulty. 

The  index  of  achieved  productivity  discussed  in  this  paper  does  not  address  training  costs. 
Training  costs  aie  a  major  training  resource  and,  therefore,  relevant  to  a  determination  of  training 
productivity.  However,  methods  for  determining  the  costs  of  military  training  are  complex  and  go 
beyond  the  considerations  of  this  paper.  Total  training  time,  herein,  can  be  translated  into  training 
costs  when  a  method  of  determining  costs  per  day  of  instruction  has  been  established.  With  such 
cost  data,  the  productivity  formulas  can  be  expressed  in  terms  of  costs  rather  than  days  of  training. 

Finally,  the  proposed  index  of  achieved  productivity  should  be  viewed  as  a  supplement  rather 
than  a  replacement  for  the  criteria  of  which  it  is  composed.  Criteria  such  as  graduation  rates,  test 
scores,  and  setback  times  can  provide  clues  to  why  the  overall  index  may  be  rising  or  falling.  Also, 
some  decisions  may  rely  only  on  a  single  index.  For  example,  for  a  given  rating,  a  fall  in  graduation 
rate  beyond  a  certain  point  may  be  considered  unacceptable,  even  if  the  overall  productivity  ratio 
is  maintained. 


CONCLUSION 

In  the  past,  Navy  training  administrators  have  sometimes  attempted  to  improve  a  single  aspect 
of  training  productivity  such  as  attrition/graduation  rates  or  setback  rates  without  considering  the 
overall  effects  on  training  productivity.  Such  focusing  can  produce  cyclical  changes  in  training 
emphasis  without  improving  training  effectiveness.  A  more  comprehensive  approach  to  training 
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improvement  is  needed  that  considers  all  available  productivity  criteria  simultaneously.  Currently 
available  productivity  criteria  include  attrition/graduation  rates,  setback '  rates,  and  test 
performance  data.  We  have  proposed  the  use  of  an  overall  index  of  achieved  productivity  that  is 
sensitive  to  the  effects  of  all  of  these  criteria.  We  have  presented  formulas  for  combining  these 
criteria  into  a  single  productivity  index.  We  have  demonstrated  how  the  effects  of  simultaneous 
variations  in  different  productivity  criteria  can  be  interpreted  through  the  use  of  such  an  index. 
Finally,  we  have  discussed  the  applications  and  limitations  of  the  index. 

The  index  of  achieved  productivity,  presented  in  this  report,  is  ai-  effective  device  for 
determining  overall  productivity  levels  of  Navy  technical  training  schools. 

RECOMMENDATION 

An  index  of  achieved  productivity  should  be  maintained  by  an  appropriate  Navy  training 
activity,  and  reponed  to  training  administrators  to  assist  them  in  assessing  the  overall  productivity 
of  technical  schools. 


i 
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